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PART I 

I. INTRODUCTION 

This part of the progress report describes the work done under 
the grant by the principal investigator while in residence at the 
Man Machine Integration Branch, NASA-Ames Research Center. It can 
only be described as a most successful year and by all measures 
fulfills the intent of the exchange agreement. The exposure to 
ideas, different approaches, and the many and varied projects 
was an enriching experience. 

Two experimental capabilities were developed on the PDP— 12 
computers located in the MMIB at NASA because of the uncertainty 
as to the status of the SEL 840/E6iS Graphics systems throughout 
the year. The first of these was an experimental monitor to per- 
form experiments in the psychophysics of visual target motion 
prediction, discrimination, etc., and exercises the (limited) 
graphical capabilities of the PDP-12. The original monitor 
(programmed by PMI) was modified by Professor Curry and Dr. Nagel 
to include the capability of handling more than 64 stimuli with- 
out reinitializing the program, and is being used in the experiment 
described in Section II. The second experimental monitor was 
programmed entirely by the principal investigator and was used 
to investigate pilot decisions in low visibility approaches. The 
details of this experimental capability, which has great potential 
for exploring decisions and behavior, is described in detail in 
Section III. 

Other accomplishments during the residence at Ames consisted 
of new analytical results in the modelling of choice behavior, 
results concerning uniqueness of parameter estimates in psycho- 
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physical models , the development of two computer programs (one 
with applicability in behavioral research, the other a general 
parameter optimization algorithm specifically designed for small 
computer applications e.g, the MMIB PDP-12) . 

II. MONITOR FOR THE PDP-12 CRT DISPLAY 

Midway through the year of residence at Ames , preparations 
were initiated to perform an experimental investigation of trajec- 
tory perception and prediction, with the dual purpose of developing 
dynamic perceptual models and determining the essential elements 
in traffic situation displays. Because of the unreliability of 
the SEL 840 at that time ^ the decision was made to perform these 
experiments on the PDP-12. The programming staff of PMI was 
given the task of developing an experimental monitor to meet these 
criteria, and work was begun in January. This was the first use 
of the automatic priority interupt capability on the PDP-12^ and 
progress was slow; the monitor was not finished until April. By 
this time our original experimental goal had to be modified because 
of the schedule slippage and the results we had developed for the 
decision behavior with multiple signal strengths (see Section IV 
below) . The experimental monitor developed by PMI is quite flex- 
ible, and the complex stimulus control and response logic have 
been debugged by PMI . The principal investigator and Dr , Nagel 
subsequently modified the monitor (to provide for more than 64 
stimuli and to allow a wait interval between each stimulus pre- 
sentation) to perform the experiments described below. 

The data in our paper are very well described by a model 
which assumes that subjects make decisions based on a subjective 
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Neyman Pearson criterion. An alternative explanation was that 
decisions are made on the basis of maximizing expected value and*^ 
that the utilities were changing with distance from the collision 
point (in violation of the subjectively expected utility (SEU) 
model). In the paper, we gave several arguments in favor of the 
subjective Neyman Pearson model as opposed to the break down of 
the SEU model, but there have been no experiments performed on 
the SEU model in a similar setting to our knowledge. In fact, 
although choice behavior has been modified by manipulating the 
payoffs in the two by two stimulus response matrix of the conventional 
signal detection paradigm, no one has actually measured the utilities 
making up the decisions in such a setting. 

To examine this and other display related aspects of the 
experimental situation (a target approaching one's own aircraft on 
a near collision course) , we set the following objectives for the 
experiments 

To measure utilities in a signal detection situation 
. To determine if utility varies with distance from col- 
lision (positive results will support the breakdown of 
the SEU model in our previous experiments - negative 
results will support the subjective Neyman-Pearson 
criterion) 

Evaluate the influence of instruction on utilities 
. Gain experience in the measurement of utilities 
From our previous experiments with this type of display , we 
know that the sensitivity (signal-to noise ratio, or d') varies as 
dg/L, where is the miss distance on the display, and L is the 
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target distance from one’s own aircraft symbol on the display. 

We have arranged a set of stimuli to vary distance and d’ in a 
factorial manner, and will thus be able to ascertain whether or 
not there is an interaction between subjective probabilities and 
untilities (distance) . We plan to evaluate the effect of instruc- 
tions of utilities by dividing the subjects into two groups: one 

group (control group) will be instructed that the experiments are 
basic research in psychology; they will be asked to extrapolate 
a line between two points. The second group (experimental group) 
will be informed that the experiment is related to traffic situation 
displays and anti-collision displays for pilots, and that they are 
to place themselves in the position of a pilot who must determine 
whether the intruding aircraft will pass to the left or the right. 

The last objective is to gain experience in the measurement 
of utilities. In the current experiments, we are using the method 
of selling lotteries (i.e., how much will the subject be willing to 
accept instead of having to play the gamble of whether he was right 
or wrong in his guess); this is a non-trivial concept to transfer 
to the subject. In the experiments described in the next section 
(decisions in low visibility approaches) we are attempting to measure 
utilities by behavioral response; the relative merits of the two 
methods will be compared at the termination of the experiments. 

In addition to the experimental monitor written by PMI, the 
principal investigator wrote auxilliary computer programs to provide 
a rapid evaluation of each subject's performance. One program is 
used to generate the pseudorandom stimuli upon which all subjects 
be tested. Immediately after the session, the subject's 
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responses (previously stored by the experimental monitor) are read 
by a program which sifts through the responses to retain only those 
which are appropriate. At the same time, these responses and the 
subject's "bets" are printed out so that they may be used in the 
latter stages of the experiment when the subject must either play 
his bet or accept the offering price. With the help of Dr. Nagel, 
a third program was written to generate an immediate indication 
of the subject's sensitivity (d') on each of the stimulus classes, 
and simultaneously calculate the criterion level used by the 
subjects (in log likelihood ratio units) . 

At the time this report is being written, the pilot subjects 
are being run to insure a smooth running experiment. The subjects 
for the data runs will be started within a week or two. 

III. DECISION MAKING IN LOW VISIBILITY APPROACHES 

In an effort which is complimentary to that of Dr. Billings and 
Dr. Lauber of the MMIB, the principal investigator developed and 
programmed a simulation which abstracts the essential elements in a 
decision making task during a low visibility approach. The purpose 
of this experimental monitor was to develop the capability to 
examine the effects of various parameters on decision making with a 
system that would be flexible and responsive to changing needs. 

This facility allows the preliminary examination of experimental 
protocols and other techniques before committing expensive simulator 
time, especially that of the airlines. In addition, it allows us 
to examine and explore methods of applying psychological stress, a 
major goal in our first set of experiments. 

A schematic of the apparatus as seen by the pilot-subject is 
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shown in Figure 1. The buttons available to the subject are RVR 
(to request an RVR reading), turn rate buttons (left, 0, or right) 
and GA, the go around button to initiate a missed approach. 

In the central portion of the CRT is a plan view of the 
approach. In the lower part of the screen are three dots corres- 
ponding to the position of the approaching aircraft (present posi- 
tion, position one second ago, and position 5 seconds ago) . In 
the center of the screen are two pairs of dots corresponding to 
the middle marker location, equivalent to the 200 foot decision 
height for a category I approach. Farther up the screen are the 
runway outline, threshold, and three pairs of approach lights or 
lead-in lights. Above that are scores posted for the results of 
any one trial; on this approach the subject would receive 100 
points for a safe landing, and -40 points for a missed approach. 

On the left of the screen is a RVR scale with two indices corres- 
ponding to 0 RVR and that for the legal minimum (2400 feet) . On 
the right side of the screen is an altimeter which has a dynamic 
range of 0 to 220 feet. The pointer indicating altitude is pegged 
at the upper right until the aircraft nears the middle marker; as 
the aircraft passes through the middle marker , the indicated altitude 
passes through 200 feet. 

A random wind disturbance from the side (correlation time of 
50 seconds) is introduced to provide a moderately-easy control 
task for the pilot. Control is maintained by pushing one of the 
three turn-rate buttons. The aircraft has the capability of 
being in either the 0 turn rate (constant heading) or a 
standard turn rate to the left or the right. The pilot's task in 
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these approaches is to "fly" the aircraft through the middle marker, 
over the approach lights, and on to the runway. (The aircraft's 
position shown in Figure 1 is close to the initial condition 
point.) Only lateral position is important, for if the pilot crosses 
the extended threshold line but is not over the runway a crash is 
recorded. If at any time before the aircraft crosses the extended 
threshold line, the pilot hits the go-around button, a standard 
rate left turn is initiated until the heading reaches 60* from 
"North" at which time the computer program assumes that a missed 
approach was made . 

The runway and approach lights may appear either to the 
right or left of the middle marker center line, and may be closer 
or farther away than the nominal position to represent electronic 
guidance errors. This is the appropriate aircraft-centered view, 
and simulates the case when one is flying the ILS with needles exactly 
centered but finds the runway to the left (or right) when break- 
out occurs, and the case when one is either high (or low) of the 
indicated altitude. 

The slant range "visibility" is included in the program, even 
though the intensity in the CRT has only two values (off, on). 

There are 5 "characters" drawn by the PDP-12 graphic system which 
are under visibility control: the three pairs of lead-in lights, 

and the right and left halves of the runway/threshold lights. 

Should the center of any of these five characters be within a 
square (centered at the aircraft position) whose half-width is the 
slant range visibility, then this character will be turned "on" 
and will be visible. The approach lights are turned off as onei 
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gets close to each pair, to simulate their passing underneath the 
nose of the airplane; this also prevents the subject from obtaining 
additional unrealistic lateral guidance information. 

A computer program was written to generate files of approach 
trajectories and currently has a catalog of nine approach trajec- 
tories. Five of these trajectories have constant (but different) 
slant range visibilities leading to the following effect: when the 

middle marker is passed, nothing is in view; soon the first approach 
light appears, followed by the second and then the third; as the 
first approach light is neared, it disappears (passes underneath), 
and then the runway/ threshold lights suddenly appear and a safe 
landing can be accomplished. The decreasing slant range visibility 

I 

in this group of five trajectories is such that one must proceed 
farther and farther beyond the middle marker (or below decision 
height) before the first approach light is sighted. The fifth of 
these five trajectories is zero-zero visibility, so the approach 
lights and runway/ threshold lights never appear. The other four 
trajectories correspond to 

(1) a high visibility approach (runway and approach lights 
are visible as shown in Figure 1 at all times) 

(2) an extremely optimistic RVR reading, but very low slant 
range visibility 

(3) passing through a fog bank after initial acquisition 

of the approach lights: the approach lights and runway 

lights "drop out", only to reappear after three to four 
seconds 

fog bank as in (3) , but the approach and runway lights 
do not reappear. 


( 4 ) 
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First Experiment 

In the first set of experiments, performed with the help of 
Dr. John Lauber, we had the following objectives: 

1, To structure the experimental setting to make the pilot 
as aversive to a crash in the simulator as he would be in 
real life. 

2. To alter the decision strategies by manipulating the 
relative values of a landing and a missed approach. 

The first objective was desirable to make the decisions as meaning- 
ful as possible. After "sacrificing" several pilots, we finally 
arrived at the following procedure. 

As the subject is led into the experimental chamber he is 
shown a poster-sized list on the wall of people who have previously 
been subjects in the experiment. Each subject is listed by name, 
organization, and score (the total number of points accumulated 
over the 50 data trials) . The first sxibject on the list was a 
fictitious one (in this case) , and in place of his point score was 
the word CRASHED in bright red letters. The experimenter writes 
in the subject's name and organization (e.g. Joe Jones, TWA) and 
leaves the score column blank. The subject is told at that time 
that should he crash during the data trials, even if on the first 
data trial, his services are no longer required. That is, in terms 
of the experiment, he is "dead". 

It was obvious to the subject at this point that he was com- 
mitted to follow through the experiment, and the idea that he 
might crash and have that event recorded for all to see had a very 
noticeable effect on almost all subjects. 
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Preliminary Results 

This is a preliminary report on the results of the experiment, 
a summary of the responses to the questionaire. A detailed exam- 
ination of the decisions in the trials themselves will be reported 
on at a later date. 

Thirteen pilot subjects participated in the test and completed 

/ 

a questionnaire, but as the simulation was changed after the first 
three pilots, they were not included in the data regarding the 
simulation itself. Of the remaining 10 subjects, 6 are airline 
pilots and 2 are IFR rated NASA employees. 

The questionaire consisted of 3 major parts: recent experience 

in low visibility approaches and missed approaches; fidelity of 
the decision simulation; and stress ratings for actual low visibility 
approaches and the simulation. The questionaire is shown in Table I. 
Recent Experience - Of the 11 pilots completing the questionaire, 

7 had made a total of 37 category I approaches within the last 12 
months (six of these 37 approaches were military approaches) . Only 
2 missed approaches were made by these 7 pilots. When asked what 
were the most comirion causes for executing a missed approach, 

(based on their experience) , the 3 most frequently mentioned items 
were 

runway alighment/crosswinds 7 times 

visibility 5 times 

other traffic 3 times 

Simulation Fidelity - The subjects were asked to comment via the 
questionaire about the simulator fidelity only with respect to 
the decision of whether or not to continue an approach. This was 
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Date? 


Position: Ca-pt. / F«0« / S,0. Cat, II Qualified? Yes / No 

Egulnment Currnetlv Flvlng: Conroanyt 

During the last 12 months, how many approaches have you flown when reported 
visibility was at or very near Cat, I minimums? 

■As Pilot: Date of most recent: 


■As Copilot: • Date of most recent: 

During the last 12 months, how many missed approaches have you flown? 

As Pilot: Date of most recent: 

As Coallot: Date of most recent: 


Judging from your own experience, what is the most likely reason for executing 
3 missed approach? 


Considering only the task of deciding whether to continue an apnroach or to 
go around, how similar is the experimental task you just flew to an actual 
approach? Mark the line below to indicate your best estimate. 

Totally Completely 

Unlike ■ Identical 

0 123 '156789 10 

Still considering only the task of deciding whether to continue an aunroach or 
to go around , what In your. opinion are the major similarities or dissimilarities 
between the experimental task and an actual low visibility approach? 


How stressful do you find actunl low ■'visibility approaches to be? 

Hot at all Extremely 

Stressful Stressful 

0 I 2 3 ij 5 S 7 8 9 10 

Kow stressful did you find the experimental task to be? 

Hot at all Extremely 

Stressful - Stressful 

0 1 2 3 'i 5 6 f - 8 9 10 

Pilot Questionnaire for Low Visibility Decision Simulation 


Table I. 
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done both on a semantic differential scale (Totally Unlike - 
Completely Identical) and by soliciting comments on the similarities 
and dissimilarities of the simulation to an actual low visibility 
approach. The ratings of the subjects are shown in Table II where, 
it is seen that the mean fidelity rating is 5.2 with a standard 
deviation of 1.87, indicating the usual dispersion in intersubject 
ratings. 

Comments on the similarities of the simulation to a low 
visibility approach detailed the assimilation of information through 
different sources (RVR, altitude, and runway alignment) . When 
commenting on the dissimilarities, 3 pilots mentioned the lack of 
danger ("one will not die if you miss", "...lacks the element of 
danger"). Two of the pilots mentioned that in a real approach more 
reliance would be placed on decision height, i.e., that is a cut 
and dried decision (a go, no-gO| situation) . Another commented that 
he felt the reward structure was not correct because in actual flight 
the rewards for going below minima may be the loss of job., etc, 
whereas reward here is a higher point count. 

There were other comments made about dissimilarites of the 
simulator and the actual approach: three pilots mentioned that 

the visual cues were different, and one pilot mentioned the fixed 
turn rate characteristics of the simulator. These were offered 
even though the question asked specifically about the similarities 
of decision making; either the questions were misunderstood or 
these factors really do influence the decision. In either case we 
feel that these latter two factors are of secondary importance in 
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Subject/ 

Organiza- 

tion 



Simulator 

Fidelity 

Rating 

Stress 

Rating 

-Actual 

Approach 

Stess 

Rating 

Simulator 

^SIM ^ACT 

®SIM 

^ACT 

1/A 

7 

3 

2 

-1 ' 

.67 

2/B 

7 

4 

6 

2 

1.50 

3/C 

3 

7 

3 

-4 

.43 

4/B 

3 

8 

5 

-3 

.62 

5/B 

7 

8 

6* 

-2 

.75 

6 /A 

4 

7 

2 

-5 

.28 

7/A 

7 

6 

6 

0 


8/C 

6.5 

5.5 

5.5 

0 


9/D 

2.8 

7 

4 

-3 


10/D 

5 

8 

6.5 

-1.5 


Mean 

S.D. 

1 

5.23 

1.87 

6.35 

1.73 

4.60 

1.73 

-1.75 

2.01 

.763 
. 344 


♦Indicated a change to 2 later in the trials 


TABLE II Semantic Differential Ratings of 
Simulator Fidelity and Stress 
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the light of the other dissimilarities mentioned by the pilots. 
Stress Ratings - The pilots were asked to rate the stress of the 
experimental task and an actual low visibility approach on a 
semantic differential scale (Not At All Stressful - Extremely 
Stressful) ; the results are shown in the other columns of Table II. 
We have added columns showing the difference in stress rating, and 
the simulator stress (rating) as a fraction of the actual stress 
(rating) . Of these 10 subjects, three felt that the simulator was 
at least as stressful as an actual low visibility approach. At the 
other extreme, is subject number 6 who reported the simulator "lacks 
the element of danger". 

Discussion 

We are very encouraged by our first attempt at inducing stress 
analogous to actual flight results, although improvements can be 
made. It is apparent that some people are not as influenced by 
the potential embarassment or "failure" as we had hoped, and 
another stressor will be required. 

One unexpected factor emerging from these experiments 
is the possible existence of airline differences. Subject 
no. 2 felt that an actual low visibility approach is a cut 
and dried decision because of the reliance on decision height. 

He alsovihdicated higher stress in the simulator than in the 
actual low visibility approach, presumably because of the lack 
of a well-defined, externally imposed decision criterion. 

Subject 5 who talked about the different reward structure in an 
actual approach (loss of job, etc) flies for the same airline 
(Airline B) . Discussions with these individuals indicated that 
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'the policy of ■tha't paarticular airline was to observe decision 
height as a hard and fast rule; descent below decision height 
was to be done under only the most extreme circumstances . in 
conversation with pilots from other airlines we found the inter™ 
pretation of the decision height to be less strict. 

Although these behavioral data are not sufficient to infer 
the existence of differences in airline operating criteria^ or 
^iffsj^®nces in an interpretation by the airline's pilots/ they 
suggest that such differences may exist/ differences (real or 
imagined) which are perceived by some pilots. 


IV. DECISION BEHAVIOR WITH RANDOMLY VARYING SIGNAL STRENGTHS 

One of the major thrusts of the research under this grant has 
been the description and modelling of decision behavior with time 
varying psychophysical stimuli. Our approach has been the extension 
of sensory- continuum models from random variables (eg. auditory 
detection) to random processes. The results of these investigations 
will have applications hot only to information and display interpre- 
tation, but to simulator evaluation as well. Interestingly enough, 
this topic has received very little attention in the literature, 
and we feel that our results are of major importance in these areas 
of application. The principal investigator developed a model for 
the description of some of our earlier data, and in collaboration 
with Dr. David Nagel of the MMIB and Mr. Gai of MIT, wrote a 
paper on our findings. The abstract of this paper, (submitted 
to the Journal of Mathematical Psychology) , is reprinted • below. 
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DECISION BEHAVIOR WITH CHANGING SIGNAL STRENGTH 

Abstract 

The Theory of Signal Detectability (TSD) has nearly 
replaced classical notions of the threshold because of 
its ability to separate sensory and decision processes 
in weak signal detection and recognition paradigms. The 
primary emphasis of recent work has concentrated on the 
sensory rather than the decision aspects and almost all 
work has been exclusively at one signal strength. We 
propose a model to describe behavior at different signal 
strengths based on subjective rather than objective 
distributions . The model predicts ensemble performance 
at a constant objective likelihood ratio (LR) criterion 
(even though subjective distributions are the basis for 
determining cutoff criteria) unless the observer adopts 
a subjective Neyman-Pearson objective. Results from an 
experiment in visual discrimination show that some ob- 
servers in fact operate at a constant objective LR's as 
s-ignal strength is varied randomly over a wide range. 

The objective LR's of the other subjects changed dramat- 
ically with signal strength, but this behavior is con- 
sistent with the use of a subjective Neyman-Pearson 
decision rule and the linear relation between subjective 
and objective log LR's found in studies of subjective 
probability. 

V. MUNOML - A MULTINOMIAL MAXIMUM LIKELIHOOD 
PROGRAM FOR BEHAVIORAL RESEARCH 

Much of the data taken in behavioral research is grouped data, 

that is, responses are grouped into categories (e.g., the familiar 

stimulus-response matrix). When developing models to explain 

these data, one must develop analytical expressions which are 

theoretical predictions for these probabilities. During the tenure 

at NASA Ames, the principal investigator developed a very general 

"executive" program to perform maximum likelihood estimation of 

the parameters imbedded in the theoretical probabilities. The 

program derives its generality from the fact that only the theoretical 

probabilities P. .(x) and the partial derivatives 3p. ./^x, are 

required for the iteration process, and these are provided by a 

user-supplied subroutine. The introduction and summary of a 
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paper prepared for publication (and NASA CR) which describes the 
program in detail is provided below. 

?flUN0ML: A MULTINCMIAL MAXIMUM LIKEI.IHDOD PROGRAM FOR BEHAVIORAL RESEARCH 

Introduction and Summary 

In our research on modelling sensory and decision phen- 
omena we were soon confronted with the task of evaluating 
both old and new models using both old and new data. Rather 
than design an ad hoc estimation program for each new model, 
as is typically done, we developed an "executive program 
which provides a general method for estimating parameters 
and simultaneously provides flexibility for accomodating 
new models with a minimum amount of programming. Our experi- 
ence with canned computer programs has been equivocal, so 
we decided to provide only the general fr^ework and let the 
user accomplish the objectives of estimating parameters for 
his particular model by writing a new subroutine within the 
constraints of the executive program. In this paper we 
report on the method of and our experience with MUNOML, an 
executive program for Multinomial Maximum Likelihood 
Estimation. 

The most common class of distributions for which para- 
meters must be extracted are multinomial distributions result- 
ing from a stimulus-response classif iciation, 

responses (YES- NO or two alternative forced choice methods) , 
the method of successive categories (rating scales) or trans- 
ition probabilities in a Markov chain. Although a number of 
methods exist for estimating such parameters (Restle, 19^1) 
we have chosen the Maximum Likelihood method and have imple 
mented the scoring of Rao to adjust the parameters from one 
iteration to the next. We have chosen the Maximum Likelihood 
(ML) method because (1) it is a member of the class of 
sistent asymptotically normal estimators (CAN); (2) it will 
easily handle situations in which all the responses fall 
into one category; (3) there are many situations in which 
the Maximum Likelihood estimator can be shown to yield 
unique estimates for parameters (but, see Curry, 1974a, where 
it is shown that other estimation techniques may have this 
property as well) ; and (4) it is the only one exhibiting 
first order efficiency (Rao, 1973) . 

The remainder of this paper is organized as follows: 
in the next section, we present the theoretical basis for 
the program, i.e. the most general functions that can be 
performed by MUNOML, the executive program. In Section 
III, we develop the expressions for some specific behavioral 
models in Signal Detection/Recognition, and in Section IV 
we briefly describe MUNOML and the method of operation. 
Section V discusses some conclusions based on our experi- 
ence with MUNOML and addresses the problem of storage- 
limited applications. The Appendices contain a FORTRAN IV 
listing of MUNOML, a listing of the subroutine to obtain 
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parameter estimates for the method of successive categories, 
and an index of program variables for MUNOML. 


VI. A RANDOM SEARCH PROGRAM FOR LABORATORY COMPUTERS 
Another program developed under the grant has wide application 
to small laboratory computers such as the PDP-12, in the MMIB. The 
algorithm takes advantage of the assets of tl:^ small computer: data 

compatibility# low operating costs, computer availability, and is 
an easy-to-use program for parameter estimation, model fitting, 
curve fitting, generalized least squares, etc. it should find 
wide usage among investigators using small computers. The abstract 
of a paper describing the algorithm is presented below. 

A RANDOM SEARCH ALGORITHM FOR LABORATORY COMPUTERS 

Abstract 

The small laboratory computer is ideal for experimental 
control and data acquisition. Post experimental data 
processing is many times performed on large computers because 
of the availability of sophisticated programs, but costs 
and data compatibility are negative factors. Parameter opti- 
mization, which subsumes curve fitting, model fitting, para- 
meter estimation, least squares, etc. , can be accomplished 
on the small computer and' offers ease of programming, data 
compatibility and low cost as attractive features. A pre- 
viously proposed random search algorithm ("random creep") 
was found to be very slow in convergence. We present a new 
method (the "random leap" algorithm) which starts in a 
global search mode and automatically adjusts step size to 
speed convergence. A FORTRAN executive program for the 
random leap algorithm is presented which calls a user- 
supplied function subroutine. An example of a function sub- 
routine is given which calculates Maximum Likelihood Estimates 
of Receiver Operating Characteristics parameters from binary- 
response data. Other applications in parameter estimation, 
generalized least squares, and matrix inversion are discussed. 

VII. SUFFICIENT CONDITIONS FOR THE UNIQUENESS OF 
PARAMETER ESTIMATES IN BEHAVIORAL MODELS 

The estimation of parameters in behavioral models (or any 

model) to fit experimental data is done by the minimization or 

maximization of statistically meaningful criterion function. 
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Constituents of this function are the experimental data; assump- 
tions concerning the underlying distributions; the form of the 
model; and the form of the criterion function (Maximum Likelihood, 
etc.). When minimizing (or maximizing) a function of parameters, 
one must always be concerned with the global aspects of the 
solution, i.e., has one found a set of parameters which yields 
the global extremum? There have been many instances where one 
has found a "molehill” without realizing that a "mountain" is 
nearby. We have examined a very wide range of behavioral models 
and parameter estimation criteria and have determined a practical 
set of sufficient conditions which will insure the resulting 
parameter estimates are unique regardless of the observations 
i.e., that there are no other values to the parameters which 
yield a local extremum of the criterion function, hence the 
local extremum is a global extremum. The abstract of the paper 
describing these results is presented below. 

SUFFICIblSlT CONDITIONS FOR THE UNIQUENESS OF 
PARAMETER ESTIMATES FROM BINARY-RESPONSE DATA 

Abstract 

The proceedure of fitting parameterized models to 
experimental data is that of extrematizing a statistically 
meaningful scalar-valued vector function. The existence 
of multiple local extrema can greatly complicate the search 
for the global solution. Sufficient conditions for unique- 
ness of the parameter estimate are usually determined from 
the convexity of the criterion surface; the convexity 
properties are determined by the statistical criterion, the 
structure of the model, the underlying distribution, and the 
observations (data) . In this paper we seek the combinations 
of criteria, models and distributions which yield sufficient 
conditions for unique parameter estimates regardless of the 
observed binary-response data values . 

Under mild sufficient conditions usually satisfied in 
practice, the Maximum Likelihood, Minimum Chi Square, and 
Minimum Transform Chi Square criteria are convex functions 
when the parameters appear linearly. These results are 
applied to equal-variance models of signal detection/recog- 
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nition, sequential response, and additive learning models 
with implications on the experimental design. Unequal- 
variance models and models of discrete-sensory processing 
{rectilinear ROC Curves) lead to nonconvex criteria for 
some observations (saddle-points are demonstrated) . Al- 
though convexity cannot be assured for these cases, the ; 
results suggest an efficient search procedure in a lower 
dimensional subspace to find global extrema. The extension 
of these results to more than two response levels is 
discussed. 



PART II 


This portion of the report describes the work accomplished at MIT 
during the reporting period. 

VIII. PSYCHOPHYSICAL MODELS OF SIGNAL DETECTION 
WITH TIME VARYING UNCERTAINTY 

Introduction 

Signal detection theory has been extensively used in the last 
two decades by psychophysicists for the study of perception and 
cognition (Swets 1973) . The principal appeal of the theory is its 
ability to separate the detection process into its two components, 
namely the sensory process and the decision strategy. However these 
two processes have not received equal attention, since most of the 
published work concentrates on the analysis of the sensory processes 
(Green and Sv/ets 1956). There are, though, many detection processes 
in which the decision strategy is at least as important as the 
sensory process e.g. when determining percent correct. In those 
cases, the most important question to be answered concerns the 
way in which the decision maker changes his criteria when the signal 
strength changes. Therefore in order to analyze decision strategies 
one has to contend with detection processes in which the signal 
strength (detectability, signal to noise ratio, uncertainty) is 
time varying. 

Two examples of such detection problems will be discussed. 

The first one is a pilot using a traffic situation display to avoid 
collisions with intruders in his airspace- The second deals with 
pilot monitoring of an automatic landing system. In both these 
cases the detection task becomes easier as the distance to the 
target decrease^ thus the signal strength can be considered as 



this class of detec- 


time varying. 

Since little prior work had been done in 
tion problems, some preliminary experiments were necessary. A 
visual discrimination experiment was designed in which the sign 
strength was changed randomly to avoid correlation between succesive 
decisions (Curry 1973) . The main conclusion that was drawn from 
these results is that the decision mater changed his threshold 
with the Change in the signal strength. This conclusion could not 
be predicted on the basis of classical SDT results, although similar 
results were reported (but not discussed) by Klnchia and Smyser 
(1967) . several decision rules that might explain the relationship 
between the threshold and the detectability were suggested. These., 
include the Neyman Pearson (N.P.) decision strategy, and the linear 

relation between threshold and detectability (Gai and Curry 1973) 

c-r:- .4 M t> G-f-rateov with subjective rather than 
as well as the modified N.P. strategy witu j 

objective probabilities (Curry et al 1974). 

After obtaining ideas about the decision strategies that 
might be used by human observers, we applied our concepts to more 
realistic situations. Two types of such situations were considered, 
in the first, we studied the effect of correlation between successive 
discrete decisions when the signal strength was changed in a 
.eguential (not random, manner, in the second, we dealt with the 
detection of a change in the mean of a stationary continuous 
stochastic process. The work on these two problems, which will 
be described in more detail in the next sections, was the mam 

effort during the last year. 

.....a.tlai Change of ^ MasL^trena^k^r^^ 

in our basic experiments one of the experimental design goals 



was to change the signal strength so that the subject's decisions 
in successive decision intervals would tend toward statistical in- 
depence, However, in many real-life situations the signal strength 
does not change in a random way, so that a correlation between 
successive decisions is almost inevitable. This raises the question 
as to whether the correlation in signal strength changes the per- 
formance of the subjects, and if so, in which direction? The use 
of SDT is particularly helpful because it can separate the effect 
of the correlation on the sensory process and the decision strategy, 
and thereby simplify the analysis. 

There are many possible ways of changing the signal strength in 
a correlated manner. The method that we chose (referred to as a 
"sequential” change) is related to the problem of avoiding collisions. 
If two airplanes are flying in linear motion with constant velocity, 
the distance between them changes linearly with time. If in addition 
one of the pilots is using a Traffic Situation Display, which is 
updated by radar (once per 4 seconds) , then the position of the 
intruder changes linearly on the display, and the state of the world 
(closest approach^ either inside or outside the miss-distance circle) 
is the same for all decision intervals. Therefore we define 
"sequential" presentation as follows; The input data is presented 
to the subjects in blocks, each one of these blocks contains a 
fixed number of decision intervals, with the following characteristics 

1. The true state of the world is the same for all the decision 
intervals within the same block. 

2. The signal strength in each interval in the block is constant, 
but is increasing from one interval to the luext. 
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3. There are no blanking periods between intervals. 

By letting the subjects make decisions in two similar tasks, 
one in which the signal strength is correlated, and one in which 
the signal strength is random, we analyzed the effect of the signal 
correlation alone. The results showed that there was a significant 
difference in the overall behavior of the subjects. However, the 
statistical test showed that there was no significant difference 
in the sensory sensitivity, i.e. the hypothesis that the detec- 
tability d’ was the same for both presentations could not be 
rejected. This means that almost all differences in behavior are 
due to changes in the decision strategy. 

In order to analyze these changes in the decision strategy, 
we used a Markov model, in which we assumed that the current 
decision' is dependent only on the previous decision (but not on 
the state of the world, or the signal strength) . The transition 
probability matrices based on this model, showed a strong tendency 
q£ the subjects to repeat their previous decisions even in those 
cases in which their decisions were incorrect in the previous in- 
terval. Therefore the Decision Rule (DR) curves for the random 
and sequential signals were totally different as can be seen in 
Figures 2 and 3. In the framework of classical SDT these results 
show an over confident behavior, in which the subject moves his 
Qj^iterion in such a way as to increase his probability of hit and 
therefore increases his probability of false alarm. The manner 
in which he does this depends on the decision rule that he is 
using. If he is using a constant likelihood ratio decision 

rule, he increases the apriori probability of that state of the 




FIGURE 2. CONDITIONAL DR CURVES FOR SEQUENTIAL AND RANDOM PRESENTATION 

SUBJECT A.C. , ' 

(STIMULUS: R=RANDOM, S=SEQUENTIAL!) 




0.25 0.50 


PCA/LO 


FIGURE 3. CONDITIONAL DR CURVES FOR SEQUENTIAL AND RANDOM PRESENTATION 

SUBJECT A.T. 

(STIMULUS: R=RANDOM, S=SEQUENTIAL) 
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world on which he had decided before. If he is using a N.P. 
strategy/ he decreases the probability of a false alarm. (See Fig 2,3). 

The conclusion from the above discussion is that the effect 
of correlated decisions is manifested in the criterion level for the 
experiments described here. 

Dete ction of a change in random processes 

In the previous section we discussed a detection problem in 
which decisions were of a discrete-time routine/ i/e. at the end 
of each decision interval. This was possible because the information 
updating was discrete (radar sweep) and as a result the (displayed) 
signal strength remained fixed for 4 seconds. In other cases the 
changes may occur continuously, as for example in the case when 
an ILS system is used for updating an automatic landing system. 

In these cases the signal strength is a continuous stochastic 
process, and the detection problem is usually a problem of failure 
detection, e.g. , a detection of a change in the steady state (s.s.) 
mean of the process. 

In order to analyze the behavior of the decision maker in 
such a situation we designed an experiment in which the subject’s 
task was to detect a change in the s.s. mean of a Gaussian process. 
The stimulus was a horizontal line (on a CRT display) whose dis- 
placement was determined by the output of a second order, time 
invariant system driven by white Gaussian noise. The steady state 
mean value of the output was zero for the non— failure mode. After 
the subject was trained and familiarized with the nominal process, 
a change in the mean was made at an arbitrary time, and the sub- 
ject had to decide whether the change in mean was up or down. 
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Ther© were four levels of changes of the means with sizes ^1/20, 

+ 0 , +20, and +30, where 0 was the standard deviation of the 
displayed process. 

Two observation intervals (limited and unlimited) were used. 
When the sxabject had an unlimited observation interval^ results 
showed that the product of the size of the change and the average 
time to detection was constant j suggesting an integrative process. 
However when the length of the decision interval was fixed and 
the subject was told that the change always occurred in each 
interval, the above relation was not kept and the average detec- 
tion time for the smaller changes in mean value was much smaller. 

The model which was suggested by the data for the description 
of the behavior is based on optimal estimation theory (Kailath 
1974) and sequential hypothesis testing (Wald, 1947) . A block 
diagram of this model is shown in Figure 4 , The displayed output 
is the input to the decision mechanism. Based on. these outputs, 
the optimal estimates for the states of the shaping filter are 
found by the use of the Kalman filter. However for the detection 
process the subject uses the filter residual to obtain uncorrelated 
measurements rather then the estimates of the state. Since we 
assume that the subject is familiar with the "non failure" mode, 
the filter is the correct filter for this mode, and is in the 
steady state. Therefore the residual is a zero mean Gaussian pro- 
cess. When a failure happens^ the mean of the displayed input is 
changed, and this will cause the mean of the residual to change. 

The detection is therefore done by a discrimination of two Gaussian 
random variables with equal variances but different means. 



FAILURE 


DETECTION MECHANISM 


SYSTEM 


1 

DYNAMICS 1 





FIGURE 4. A MODEL FOR DETECTION OF A CHANGE IN THE MEAN OF A RANDOM PROCESS 
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This discrimination is done by using Wald’s sequential prob- 
ability ratio test (Wald 1947). The decision function is the sum 
of the successive likelihood ratio of the residuals. However because 
of the finite memory of the human subject he does not use the 
entire residual history and we have included an exponential 
smoothing operator (Schweppe 1973) . 

The decision mechanism is somewhat different from the decision 
mechanism that is used in classical SDT. In sequential observations 
the subject chooses two thresholds A and B and decides: 

^^if the decision function is greater then A 
Down ^ if the decision fionction is smaller then B 
Take another measurement ^ if the decision mechanism 
is between A and B 

i 

The values of A and B are determined by the values which the sub- 
ject assigns to the two types of error. For a free length 
interval the values for the two types of error are fixed during 
the whole decision interval. However when the length is fixed 
and because the subject knows that a change must occur, he tends 
to let the type I error grow with time. 

The average detection times for two subjects as a function 
of the change in the mean for a unlimited decision interval are 
shown in Figures 5 and 6. The results predicted by the model 
(disregarding the finite memory) are also shown in those figures. 
Figures 7 and 8 show the same results for the same subjects in a 
limited decision interval experiment. , The predicted results of the 
model, in which the size of the error of type I is changed expon- 
entially with time are also shown on these figures. 
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FIGURE 5. EXPERIMENTAL AND OPTIMAL DETECTION TIME 

SUBJECT B.C. 
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FIGURE 6. EXPERIMENTAL AND OPTIMAL AVERAGE DETECTION TIME 

SUBJECT A.C. 
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FIGURE 7. PREDICTED AND MEASURED AVERAGE DETECTION TIME FOR CLOSED 
INTERVAL SUBJECT B.C. 
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0.03 0.06 0.12 0.18 

FIGURE 8. PREDICTED AND MEASURED AVERAGE DETECTION TIME FOR CLOSED 
INTERVAL SUBJECT A.C. 
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IX. FAILURE DETECTION BY PILOTS IN MULTI-AXIS TASKS 
Background 

This study of failure detection or the decision of proper 
system operation by the pilot concerns itself with the situation 
of an aircraft in an automatic Category III landing approach. 

The pilot monitors the progress of the approach and the operation 
of the equipment and provides a (hopefully) failsoft capability: 
should the automatic landing system (ALS) fail, the pilot will 
detect the failure, identify it and take corrective actions as 
dictated by the type and time of the failure. 

It is axiomatic that the pilot should be capable of detecting 
and identifying failures of the ALS accurately, reliably, and with 
minimal time delay. We hypothesize that the factors with signifi- 
cant effect on the pilot’s failure detection capability are the 
following 

a. Participation — ranging from passive monitoring of the 
displays to actively controlling in one or more axes. This 
appears to be important because Young (1969) found that monitors 
have poorer detection performance then controllers; Vreuls et al 
(1968), on the other hand, found monitors to be better failure 
detectors . 

b. "Workload” — induced by the primary task(s) and the associ- 
ated disturbances, by secondary tasks, and by variations in con- 
trol dynamics. 

Work to date 

Simulation facility — During the past months, a simulation capability 
including the ADAGE AGT/30 digital graphics computer and a fixed- 
base cockpit simulator have been developed. 
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a. Simulator dynamics — A mathematical model has been developed 
of a large transport aircraft. The actual flight data of a DC-8 
were used in the equations of motion (Teper 1969) and the various 
parameters were later refined following a series of flight tests 
by an American Airlines senior captain. The flight envelope of the 
simulator ranges from landing approach to cruise at up to 400 kts., 
at altitudes from 0 to 6000 feet. Non-linear phenomena such as 
ground effect and stall characteristics have also been included. 

The simulator has been flown by ex-fighter pilots and airline 
captains and all feel it is more than adequate for the experimental 
program to follow. As an illustration: a non-experienced pilot 

tried to align the aircraft with the runway center line on short 
final by skidding at a high angle of bank; the simulator reacted 
by entering an over-the-top flat spin. 

An integrated cue flight director system has been designed 
for this simulator, providing the capability to land it manually 
in zero-zero conditions in a satisfactory manner. Also, a two- 
axis autopilot has been incorporated into the simulation which is 
capable of flying ILS-coupled approaches, in either axis or in 
both axes, to touchdown. The autopilots and the flight director 
system have been tested extensively. 

We also have the capability to add wind disturbances to the 
simulation. The current wind modes are; 

a. No wind. 

b. Steady 10 kt. wind from 260° (i.e., at 135° to the 
runway heading. Runway 4R at Logan Airport, whose 
heading is 35®, is our active runway). 
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o. 5 kt. wind, 9 “Sting to 10 kt., from 260 
d. 10 kt. wind, gusting to 20 kt. , lrom260°- 
The gusts are modelled as filtered white noise with a cutoff fre- 
quency of t/6 rad/sec. (See Appendix A, “Disturbances") 

b. Displays-A CRT mounted on the oaptain's instrument panel is 
used to present flight information in the format of conventional 
instruments: airspeed, attitude and flight director, DME, vertioal 
speed, HSI, EMI, altitude and localizer/glide slope deviatio 
we also have the capability to incorporate a flight director mode 

annunciator, if desired. 

c. support software-support programs have been written to 
input analog data (oontrol column, rudder pedals , etc.), discretes 
(e.g., gear), output (CRT, marker beacon indicators , etc.) to operate 
the side wsk and to store trajectory data in real time (see Appendix 

B, "Sample of Trajectory Data ). 

The side-task is of the warning light type: two small red 

lights are mounted olose to each other in the pilot's peripheral 
vision field. Either one of the lights turns on at random times , 
uniformly distributed between 0.5 and 5 sec., and stays on for 2.0 
secs, at most. The pilot is required to turn the light off with a 
three-position, spring-loaded rocker thumb switch which is mounted 
on the left horn of the control wheel. The program records a correct 
response if the switch is activated in the correct direction within 
the 2.0 seconds: it also stores the response-time and the spatial 

coordinates of the aircraft at the time of the response and turns 
the light off. An incorrect response is recorded, and the spatial 
coordinates stored if the switch is activated in the wrong direction 
or if the light has not been turned off within the 2.0 seconds. 
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This side task is similar to the one used by Spyker et al 
(NASA CR-1888 1971). It is especially suitable for our purposes 
because it reportedly does not require the pilot's entire reserve 
capacity and therefore does not result in a significant degradation 
of primary task performance (5% loading was reported); at the same 
time the performance on this side task is highly sensitive to atten- 
tion on the primary task, making it a good workload measuring device. 
Future Work 

The purpose of this research is the study of the pilot's short 
term decisions regarding performance assessment and failure monitoring. 
We wish to investigate the relationship between the pilot's ability 
to detect failures, his degree of participation in the control 
task, and, his overall workload level. To this end, the following 
three phases of work will be undertaken; 

a. Completion of the simulation — As we already have a good 
simulator incorporating autoland capability, several levels of 
wind disturbances and workload measuring side task, we only need 
to add pre-programmed failures in the lateral and longitudinal 
axes. During the development of the simulator we had many unplanned 
failures which we may now incorporate into the programs deliberately. 

b. Experimentation — In this phase, scheduled to last for 
four to six months, airline pilots who are type -rated in either 
B-707, DC-8 or B-747 will be asked to fly approaches with different 
degrees of automation and with different levels of wind disturbance; 
the pilot's ability to detect failures, to correctly identify 

them and to provide a reliable manual back-up capability will be 


monitored. 
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c. Analysis—The data recorded in the second phase will be 
analyzed, to identify statistically significant relationships among 
the experimental treatments, to wit, participation, workload and 
failure detection. An optimum point will be sought, i.e., the 
participation mode and workload level which produce the optimal 
failure detection performance. Equally important is the sensitivity 
of failure detection to these independent variables. 
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APPENDIX A 

Disturbances 

Both horizontal and vertical disturbances are modelled as 
random wind gusts. A random numbers generator is therefore 
incorporated in the program, as follows: 

Define = (7701 • + 3927) mod 10,000 \ - 7129 


is then a random number in the range 0 ^ < 10,00Q, 

n 

with the probability distribution 



to obtain X^, a random number in the range -I* 1 ^ with the 

square distribution 
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The gust sequence y was obtained from the random sequence 
by passing it through a first order filter G(s) 


G(s). = 


_ 1 


S+O) . 


The output of the filter was sampled by the program at intervals 
of T seconds (the program's update rate), to obtain the gust 
sequence 

y n+l = ^ ^0 = “ 

The gust sequence has the following statistics: 


= T • but X = 0 y = 0 


^(JL) « X ,-k 4^ ^ 

2 i 2 „2 2 

a=e ’a+Ta 

y y X 


or 




-2o)^T 


It is desired that the gusts y^ should not exceed some :preset 
value 99 .75% of the time (which corresponds to 3a^) . There- 

fore, Oy = V^^^/3 and 


„ max 

^x “ 3 


^l-e~ 


2w^T 


T 


w , -2oj.T 
js, „ _ /12 ^ _ /12 '^max ^1-e ^ 


v_ 


max 


■V 


1-e 


-2o).T 
1 
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To suitmiarize ; 

1. We generate the random numbers sequence 


X = {7701 • X + 3927)mod 10,000 = 7129 

n+1 n 0 


2. Define the modified random sequence 


/ ^ 

X = f ^ 
^n ^ 10,000 



max 


1-e 


-2w.T 
1 


\I 


where; V is the desired maximum gust velocity 
max ^ 

is the gust's cutoff frequency set at tt/6 rad/sec 
T is the program's update time, ^0.2 sec 

3. Pass this sequence through a first order filter G(s) 


G(s) = 


s + 


to obtain the random wind gusts. 

4. As a final step, a steady (constant) wind is superimposed 
on the gusts. 


In a series of tests in the computer, the actual mean and 
standard deviation of the generated gusts were found to be 
within less than 1.2% of the theoretical values, even when as 
little as 150 sample points were used. 

Dynamics 

The aircraft is assumed to posess two separate motions; 

a. motion relative to the air (wind axes) . 

b. motion of the air relative to the ground. 

The vector addition of these two motions yields the motion of 
the aircraft relative to the ground. 

Also , if there is a component of the wind normal to the 
heading of the aircraft, the aircraft is assumed to acquire 
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a component of velocity, relative to the ground according 

_ T/t 

to the relationship V„ = V (1-e ' ) or 

n 


V„(s) 


= yt 

s + 1/t 



from the geometry of the problem; 

% " ^0) * (j^ - p 

^nm = * sin(j|i^ - p 

^nx = -Vj^cos[90° - ( 1 - 35 -)] = -V^sin {j|;-35° ) 
'^ny = V^cos(^-35°) 
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wihere, in the ground frame of reference: 

V is the wind velocity component colinear with the air— 
craft heading 

^nu is the wind velocity component normal to the aircraft 
heading 

is the aircraft velocity component normal to its heading 
induced by the wind. 

V , V are the components of along the x,y axes, 

nx' ny n 

Now, 





(s) 


1 

ts+i 


=> = llv^n-Vnl = - ''nJ 


The aircraft's ground speed, V , is then computed from its 

y 

airspeed: 


cos (i(> 
-Ho 


- tp) 


and the aircraft senses a side-slip angle 3 


V - V 

^ o ... no) n 

3 = + arc tan ^ 

a 

The components of the aircraft's ground speed along the principal 
axes are: 

= [V^ - i|^) ] *cos (i|^ - 35®) - • sin(i{^ - 35®} 

= [V^ + • cos (jj^ - ^)]*sin(^ - 35°) + • cos - 35°) 
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For the purpose of the simulation, the following values are 
used : 

t = 1 sec 

i]j =80° (45° to the runway heading, which is 35°) 

— 0 ) 

and three values for V^; 

a. V =10 kts. steady wind 

(a) 

b. V = 5 kts. steady wind + gusts ranging between +5 kts. 

c. V =10 kts. steady wind + gusts ranging between +10 kts. 

w 



Gust Velocity 
(Knots) 

. .£5,000 


/ 





^^AN = 0,15 ST^NT>A??D D^VIf^TICN = 3,37 

m:< ^ 10,00 cuTorr FREQUDicv • =0.524 



P^ESS SWITCH 4 TO CPDATE, ??nTCH 1 TO CXIT,,, 
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APPENDIX B 


SAMPLE OF TRAJECTORY DATA 








Instrument Panel 


Figure 10. 
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PARAMETERS AT TOUCHDOWN OR AT STOPACTION 


DISTANCE FROM THRESHOLD 
DISTANCE FROM CENTERLINE 
INDICATED AIRSPEED 
VERTICAL SPEED 
FLARE COMMANDED AT ALT. 


PITCH ANCLE 

5. 

OEGS 

BANK ANCLE 


Decs 

HEADING 

35. 

DEGS 

GROUND TRACK 

3S. 

DEGS 

CRAB ANGLE 

1 . 

DEGS 

DT 

= D.2000 

DATA UPDATE RATE 


5 

LOCD= S7 HITS= 73 

Ml S3 

= 12 



1707. FT. 
12. FT. 
127. KNOTS 
“178, FPM 
45.9 FT. 
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1 6 C 2 • 
1553. 
1425. 

1 42 5 . 
1359. 

I 3f 9 . 

1 o r.c ' 

12fc4. 

1 1-,2V 
12 2 1. 
i::2r. 

966V 

9 12 . 

2 55V 
7 V 4 . 
735. 
£77. 


Y, FEET 

-6279. 

-5347V 

-4c37. 

-3926. 

-32er. 

-2496 . 

- 1 £4C . 
-1295. 

■ -310. 
-432. 
-124. 

■ 9cV 
259 V 
361. 
412 . 
429 . ■ 
4 1 S . 

. 39£ . . 
347. 
3£ 3 V 
26£ . 

£ I 6 . 
175. 
139. 
157 . 
79 . 
57 . 
41 . 
29 V 
25 • 
13. 

6 . 

4 , 

2. 




- 1 V 
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Elapsed Time 

!^!3V .Cir; 

2 5 r • 3 9 
2 (3 V ♦ i o 3; 
225* 137 
27 2'*/ 37 
275*336 • 

2 3 ■': • 2 75 
235*22A 
2C: .333 
£9 1 • 1 33 
292 • 133 
29 3.3 33 
2 9/:.3£E 
29 5. 132 

cl VV..' • I 

29 7 .'2 32 
29 f .732 
299.131 
3C2. 131 
35' 1 . 1 £ I 
3C2 .331 
333'.C31 

'5 ,332 


326 . 1 3C 


X/ feet 

- 1962 3. 


‘C/ll? ' 
■7 32 Z- 
• c 2 -4 9 ' 
■ 51 -'i 2 


~29 1 7’. 
-1C5 V. 
-1623 . 
-1393. 
-1297V 
-977V 

-ni\i • 

-516. 


'7 

- 1 C 3 . 
152. 
361 . 
ict' . 

766. 

n ’ 

> O C . 

1216. 

U736V 

1 664 . 


A, feet 

629. 

56 5'. 

52 6. 
446. 

3L V . 

33 1 V 
274 . 
215. 

153 . 

1 4 6 . 
123 . 
123. 
111 . 
99. 
27. 
77. 

6 o • 


33 . 
5 5* 

16 . 

5 . 
4 . 

CV 


Y, feet 

-1 . 

- 1 . 

- 1 . 

-IV 
■- 1 V 


-GV 
-£ V 





— 6 . 



C7> tSl tV'i O r:'! 
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44 COI'PZCT F,“£rOKS-S FOLLOU: 


X, feet 

P2SF0K 


-?£4£9 


532 

2 

.22C 

•• 6 969 3 


872 

e 

.22£ 

-68662 


2 3 2 

e 

.acE 

-67578 


O C/? 

e 


-62242 


ir 

s 

r> nt 

-58873 


C6C 

2 

.250 

— 5 c -r6 3 


312 

G 

.2££ 

-57 272 


782 

e 

.2EE 

-56772 


47 2 

£ 

.232 

-54772 


372 

2 

.2 2 2 

- 5 3 9 8 1 


£40 

e 

.2E£ 

- 53494 


292 

e 

.2CC 

-52475 


812 

0 

.2££ 

-45316 


375 

e 

.23E 

-44725 


'ij't c; 

e 

.2EG 

-44287 


C* 1 •i' 

c 

.aoe 

-42574 


43 5 

e 

.2CG 

-42644 


312 

G 

.22£ 

-42334 


722 

e 

.2££ 

-38223 


0^!7 
C 4^ 


.2EG 

-53443 


162 

0 

» 2 50 

-33125 


42 5 

0 

.2£2 

-31433 


52 1 

e 

.2 5-2 

-26356 


248 

e 

.2GG 

-26239 


829 

0 

» 2 £• '3 

-22557 


719 

2 

.£££ 

-19432 


516 

0 

• 2 5£ 

-17254 


125 

0 

.222 

-166G7 


328 

£ 

.222 

-14575 


37 5 


.222 

-1433 5 


35 1 

£ 

.22£ 

- 1 2 52 8 


272 

G 

.2£G 

-1 1 543 


414 

2 

.222 

- 1 2 5 5 1 


46 5 

£ 

.223 

■ -9388 


125 

£ 

.222 

— 0153 


66 4 

E 

.222 

-862 5 


62 5 

e 

.222 

-787 1 


250 

G 

rs /■« 

« ci fc. c; 

-7323 


219 

£> 

.223 

- 6 52 3 


29 7 

G 

.252 

-4285 


554 

e 

.£££■ 

-392 3 


252 

£ 

.222 

-31 1 4 


224 

0' 

.2CC 

- 143 7 


£33 

0 

.253 
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ZZT PESFOrjSE 
feet 


••672 52 • o 
-66772.3 
-63976.4 
-63457.4 
-64162. I 
- 6566 1*9 
-63237 .e 
-62526. £ 
-62c44.9 
-62303.3 

-scisr.s 

-57612.6 

-561£ 1 .4 

-5555f.e 

-52216.9 

-51671 .4 

-47783.2 

-472 38. 5 

-46137.7 

-45627. 1 

-41604.8 

-41123.4 

-38696.0 

-38402.7 

-36894V0 

-36442'.4 

-3 570 8’. 9 

-35282'. 5 

-34340.0 

-33921 '.2 

-32657'.0 

-32239.8 

-30 692'. 5 

-30275.8 

■-29946V8 

-29530'. 3 

-28872'.4 

-28455'.8 

-27568.2 

£27141'.0 

-25420.8 

-24993.3 

-24532.9 

-24105.2 

-219C9.0 , 

-21572 . 5 

-21287.5 

-20882.0 

-20049. 1 

-19632.7 

- 18350 . 3 

-17933.8 


rcLLoi:; 



X, feet 


156t6.5 
15269.9 
13932.8 
13516.3 
-5558.5 
-5141.9 
-2686 . c 
-2278. E 




RWY MM OM 


FT X 10 
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Curry, R.E. , Nagel, D. Gai, E.G. (1974) Decision behavior 
with changing signal strength. To be published in 
the J of Mathematical Psychology. Also presented at 
the Tenth Annual Conference on Manual Control, 1974. 

Curry, R.E. (1974) Sufficient conditions for original 

parameter estimates in behavioral models. In preparation. 

Curry, R.E. (1974) MUNOML: A multinomial maximum likelihood 

program for behavioral research. In preparation. 

Curry, R.E. (1974) A random search algorithm for laboratory 
computers. In preparation. 
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